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Abstract 

We consider the Cascade and Triangular rate-distortion problems where the same side information is available at 
the source node and User 1 , and the side information available at User 2 is a degraded version of the side information 
at the source node and User 1. We characterize the rate-distortion region for these problems. For the Cascade setup, 
we showed that, at User 1, decoding and re-binning the codeword sent by the source node for User 2 is optimum. 
We then extend our results to the Two way Cascade and Triangular setting, where the source node is interested in 
lossy reconstruction of the side information at User 2 via a rate limited link from User 2 to the source node. We 
characterize the rate distortion regions for these settings. Complete explicit characterizations for all settings are also 
given in the Quadratic Gaussian case. We conclude with two further extensions: A triangular source coding problem 
with a helper, and an extension of our Two Way Cascade setting in the Quadratic Gaussian case. 

Index Terms 

Cascade source coding, Triangular source coding, Two way source coding, Quadratic Gaussian, source coding 
with a helper 

I. Introduction 

The problem of lossy source coding through a cascade was first considered by Yamamoto where a source 
node (Node 0) sends a message to Node 1, which then sends a message to Node 2. Since Yamamoto's work, the 
cascade setting has been extended in recent years through incorporating side information at either Nodes 1 or 2. 
In |2|, the authors considered the Cascade problem with side information Y at Node 1 and Z at Node 2, with the 
Markov Chain X — Z — Y. The authors provided inner and outer bounds for this setup and showed that the bounds 
coincide for the Gaussian case. In Q, the authors considered the Cascade problem where the side information is 
known only to the intermediate node and provided inner and outer bounds for this setup. 

Of most relevance to this paper is the work in |4], where the authors considered the Cascade source coding 
problem with side information available at both Node and Node 1 and established the rate distortion region for 
this setup. The Cascade setting was then extended to the Triangular source coding setting where an additional rate 
limited link is available from the source node to Node 2. 
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Fig. 1: Cascade source coding setting 
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Fig. 2: Triangular source coding setting. 



In this paper, we extend the Cascade and Triangular source coding setting in |4) to include additional side 
information Z at Node 2, with the constraint that the Markov chain X — Y — Z holds. Under the Markov constraint, 
we establish the rate distortion regions for both the Cascade and Triangular setting. The Cascade and Triangular 
settings are shown in Figures Q] and |2] respectively. In the Cascade case, we show that, at Node 1, decoding and 
re-binning the codeword sent by Node to Node 2 is optimum. To our knowledge, this is the first setting where 
the decode and re-bin scheme at the Cascade is shown to be optimum. It appears to rely quite heavily on the fact 
that the side information at Node 2 is degraded: Since Node 1 can decode any codeword intended for Node 2, 
there is no need for Node to send additional information for Node 1 to relay to Node 2 on the i?i link. Node 
can therefore tailor the transmission for Node 1 and rely on Node 1 to decode and minimize the rate required 
on the i?2 link. We also extend our results to two way source coding through a cascade, where Node wishes to 
obtain a lossy version of Z through a rate limited link from Node 2 to Node 0. This setup generalizes the two way 
source coding result found in (5 |. The Two Way Cascade Source Coding and Two Way Triangular Source Coding 
are given in Figures [3] and [4] respectively. 



Node 




Fig. 3: Setup for two way cascade source coding. 

The rest of the paper is as follows. In sectionQ]] we provide the formal definitions and problem setup. In section Hill 
we present and prove our results for the aforementioned settings. In section [TV] we consider the Quadratic Gaussian 
case. We show that Gaussian auxiliary random variables suffice to exhaust the rate distortion regions and their 
parameters may be found through solving a tractable low dimensional optimization problem. We also showed that 
our Quadratic Gaussian settings may be transformed into equivalent settings in flD where explicit characterizations 
were given. In the Quadratic Gaussian case, we also extended our settings to solve a more general case of Two 
Way Cascade source coding. In section [V] we extend our triangular source coding setup to include a helper, which 
observes the side information Y, and has a rate limited link to Node 2. Our Two Way Cascade Quadratic Gaussian 
Extension is shown in Figure [5] (in section HVb. while our helper extension is shown in Figure [7] (in section fVV We 
conclude the paper in section [VI] 
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Fig. 4: Setup for two way triangular source coding. 



II. Problem Definition 

In this section, we give formal definitions for the setups under consideration. We will follow the notation of J6] 
Lecture 1]. Unless otherwise stated, all logarithms in this paper are taken to base 2. The source sequences under 
consideration, {Xi £ X, i = 1, 2, . . .}, {Yi £ y, i = 1, 2, . . .} and {Zi € Z, i = 1, 2, . . .}, are drawn from finite 
alphabets X, y and Z respectively. For any i > 1, the random variables (Xi, Yi, Zi) are independent and identically 
distributed according to p(x, y, z) = p(x)p(y\x)p(z\y); i.e. X — Y — Z. The distortion measure between sequences 
is defined in the usual way. Let d : X X X — > [0, oo). Then, 

1 ™ 

n £ — ' 

i=i 



A. Cascade and Triangular Source coding 

We give formal definition for the Triangular source coding setting (Figure |2). The Cascade setting follows from 
specializing the definitions for the Triangular setting by setting R3 = 0. A (n, 2 nRl , 2 nR2 , 2 nRs , D\, D2) code for 
the Triangular setting consists of 3 encoders 

X n x y n -> Mi 6 [1 : 2 nRl ], 

yn x jj . 2 ni? 1 ] ^ Ma g ^ . 2 ™^] 5 

A^" x 3^" -> M 3 G [1 : 2 nR3 }, 



fi (at Node 0) 
f 2 (at Node 1) 
h (at Node 0) 



and 2 decoders 



such that 



gx (at Node 1) : y n x [1 : 2 



nRi 



ff2 (at Node 2) : Z" x [1 : 2"" 2 ] x [1 : 2 nKs ] 



^2 ! 



1 

1=1 



< D 



j- 



j=L2, 



where If = fi(X n , Y n )) and X 2 " = g 2 (Z n , f 2 (Y n , fi(X n ,Y n )), fz(X n , Y n )). 

Given (D\,D2), a (Ri, R2, R3) rate tuple for the triangular source coding setting is said to be achievable if, 
for any e > and n sufficiently large, there exists a (n, 2"( fll + c \ 2 Tl ( fl2+e ), 2™( fl3+e ), L»i + e, L> 2 + e) code for the 
Triangular source coding setting. 

The rate-distortion region, 1Z(Di, D2), is defined as the closure of the set of all achievable rate tuples. 

Cascade Source coding: The Cascade source coding setting corresponds to the case where R3 = 0. 
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B. Two way Cascade and Triangular Source Coding 

We give formal definitions for the more general Two way Triangular source coding setting shown in Figure |4] A 
(n, 2 nRl , 2 nR2 , 2 nR3 , 2 nR4 ,D 1 ,D 2 , D 3 ) code for the Triangular setting consists of 4 encoders 

h (at Node 0) : X n x y n -> M x 6 [1 : 2 niil ], 

/a (at Node 1) : y n x [1 : 2 nRl ] — » M 2 £ [1 : 2 n/ * 2 ], 

/ 3 (at Node 0) : X n x y l -> M 3 G [1 : 2 ni * 3 ], 

/ 4 (at Node 2) : Z" x [1 : 2 nR ' 2 ] x [1 : 2 nR ' 3 } — » M 4 6 [1 : 2 nJ?4 ], 



and 3 decoders 



3 i (at Node 1) : y n x [1 : 2 niil ] ^ -Yf, 

52 (at Node 2) : Z" x [1 : 2 nR2 } x [1 : 2 nRs ] - 

g 3 (at Node 0) : X n x y n x [1 : 2 nRi ] i", 



^2 i 



such that 



1 " 

1 " 

T) Z ^ 



< Dj, j=l,2 and, 



where X™ = g\{Y n , f\{X n ,Y n )), X? = g 2 (Z n , f 2 (Y n , f x {X n , Y n )), f 3 (X n , Y n )) and 
Z n = g 3 (X n ,Y n , f 4 (Z n , f 2 (Y n , h{X n , Y n j), f 3 (X n , Y n ))). 

Given (Di, D 2 , D 3 ), a {Ri, R 2 , R 3 ,Ri) rate tuple for the two way triangular source coding setting is said to be 
achievable if, for any e > and n sufficiently large, there exists a (n, 2 n( - Rl+e \ 2 n( - R ' 2+e \ 2 n ( Rs+e \ 2 n{ - Ri+e \ D x + 
e, D 2 + e, D 3 + e) code for the two way triangular source coding setting. 

The rate-distortion region, 7Z(Di, D 2 , D 3 ), is defined as the closure of the set of all achievable rate tuples. 

Two way Cascade Source coding: The Two way Cascade source coding setting corresponds to the case where 
R 3 = 0. In the special case of Two way Cascade setting, we will use R 3 , rather than R4, to denote the rate from 
Node 2 to Node 0. 



III. Main results 

In this section, we present our main results, which are single letter characterizations of the rate-distortion regions 
for the four settings introduced in section [TT] The single letter characterizations for the Cascade source coding 
setting, Triangular source coding setting, Two way Cascade source coding setting and Two way Triangular source 
coding setting are given in Theorems Q] [2] [3] and |U respectively. While Theorems Q] to [3] can be derived as special 
cases of Theorem|4j for clarity and to illustrate the development of the main ideas, we will present Theorems Q] to [4] 
separately. In each of the Theorems, we will present a sketch of the achievability proof and proof of the converse. 
Details of the achievability proofs for Theorems QJH are given in Appendix lAl Proofs of the cardinality bounds for 
the auxiliary random variables appearing in the Theorems are given in Appendix |B] 



A. Cascade Source Coding 

Theorem 1 (Rate Distortion region for Cascade source coding): TZ(Di, D 2 ) for the Cascade source coding set- 
ting defined in section ITU is given by the set of all rate tuples (Ri,R 2 ) satisfying 

R 2 >I(U;X,Y\Z), 
i?i > I(X;X!,U\Y) 
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for some p(x, y, z, u, x\) = p(x)p(y\x)p(z\y)p(u\x, y)p(x\ \x, y, u) and function : U X Z — > X 2 sucn that 

C'/r-V-.Y ; ;- lh. j=l,2. 

The cardinality of U is upper bounded by \U\ < \X\\y\ + 3. 

If Z = 0, this region reduces to the Cascade source coding region given in PI. If y = X, this setup reduces to 
the well-known Wyner-Ziv setup J7)- 

The coding scheme follows from a combination of techniques used in fij and a new idea of decoding and 
re-binning at the Cascade node (Node 1). Node generates a description U n intended for Nodes 1 and 2. Node 1 
decodes U n and then re-bins it to reduce the rate of communicating U n to Node 2 based on its side information. In 
addition, Node generates X™ to satisfy the distortion requirement at Node 1 . We now give a sketch of achievability 
and a proof of the converse. 
Sketch of Achievability 

We first generate 2 n ^ I ^ X ' Y ' U ^ + '^ U n sequences according to Y[7=iP( u i)- F° r eacrl u ™ w< ^ 2/" sequences, we 
generate 2 n ^ I ^ Xl '> x \ u > Y )+ e ) X™ sequences according to Yl7=i P(%i\ u ii Hi)- Partition the set of U n sequences into 
2n(i(u-,x\Y)+2e) bin ^ ^ mio y Separately and independently, partition the set ofU" sequences into 2 ri ( / ( C7 ' X > y l z )+ 2e ) 
bins, B 2 (ma), m 2 G [1 : 2"( / ( c/ -^ y l z )+ 2e )]. 

Given x n ,y n , Node looks for a jointly typical codeword u n ; that is, (u n ,x n ,y n ) 6 T}^ ■ If there are more 
than one, it selects a codeword uniformly at random from the set of jointly typical codewords. This operation 
succeeds with high probability since there are 2 n ^ I ^ X ' Y '' U > + ^ U n sequences. Node then looks for a x™ that is 
jointly typical with u n ,x n ,y n . This operation succeeds with high probability since there are 2"( 7 ( Xi;X l c/ ' y ) +e ' x" 
sequences. Node then sends out the bin index mio such that u" £ Si(mio) and the index corresponding to x™. 
This requires a total rate of i? x = I(U;X\Y) + I(X?;X\U, Y) + 3e. 

At Node 1, it recovers u n by looking for the unique u n sequence in Si(mio) such that (u n ,y n ) € % . Since 
there are only 2 n ^ I ^ X ' Y '' U ^~ It ^ U;X]lY ^^ = 2 n ^ I ^ u ' ,Y ^~^ sequences in the bin, this operation succeeds with high 
probability. Node 1 reconstructs x n as x". Node 1 then sends out m2 such that u n € 82(1712). This requires a rate 
of R 2 = I(U;X,Y\Z) +2e. 

At Node 2, note that since U - (X,Y) - Z, the sequences (U n , X n ,Y n , Z n ) are jointly typical with high 
probability. Node 2 looks for the unique u n in 62(7712) such that (u n ,z n ) e 7e . From the Markov Chain 
U - (X,Y) - Z, I(U;X,Y) - I(U;X,Y\Z) = I(U;Z). Hence, this operation succeeds with high probability 
since there are only 2"( / ( c/;Z ' -e ) u n sequences in the bin. It then reconstructs using x% = 52(^1, Zi) for i £ [1 : n]. 

Proof of Converse: Given a (n, 2 ni?1 , 2 ,li?2 , £> x , D 2 ) code, define f/, = (X'" 1 , F^ 1 , Z 1 " 1 , M 2 ). We 
have the following. 

nR 2 > H(M 2 ) 
> H(M 2 \Z n ) 
= I(X n ,Y n ;M 2 \Z n ) 

n 

= J2^(Xi,Y i ;M 2 \Z n ,X i - 1 ,Y i - 1 ) 

i=l 
n 

= Y,H(X i ,Y i \Z n ,X i -\Y i - 1 ) - H(X i ,Y i \Z n ,X i -\Y i - 1 ,M 2 ) 

i=l 
n 

= H(X h Yi\Zi) - HiX^YilZ^Ui) 

i=l 
n 

= Y,I(X i ,Y i ;U i \Z i ). 

i=l 

Next, 

nRi > H(M 1 ) 

> H(M 1 \Y n ,Z n ) 
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= H(M 1 ,M 2 \Y n , Z n ) = I(X n ; M 1 ,M 2 \Y n , Z n ) 

n 

= J2l(X i ;M 1 ,M 2 \X i - 1 ,Y n ,Z n ) 

i=l 
n 

= ^2H(X i \X i - 1 ,Y n ,Z n ) - H{X i \X i - 1 ,Y n ,Z n ,M 1 ,M 2 ) 

i=l 

71 

= H{Xi\Yi, Zi) - HiX^X 1 - 1 ^ 11 , Z n , Mi, M a ) 

i=l 
n 

( = } Y, H(Xi\Yi) - H(X t \X*-\Y n , X u , Z n , M 1 ,M 2 ) 

i=l 
n 

> Y H(Xi\Yi) - HiXilXuM, Ui) 

i=l 
n 

= Y I {X i \X li ,U i \Y i ). 

i=l 

Step (a) follows from the Markov assumption X — Y — Z and the fact that Xu is a function of (Y n , M 2 ). Next, 
let Q be a random variable uniformly distributed over [1 : n] and independent of (X n ,Y n , Z n ). We note that 
X Q =X,Y Q =Y,Z Q = Z and 

R 2 >I{X Q ,Y Q ;U Q \Q,Z Q ) 

= I(X Q ,Y Q ;U Q ,Q\Z Q ) 

= I(X,Y;U Q ,Q\Z), 
R 1 =I(X Q ;X 1Q ,U Q \Y Q ,Q) 

= I(X;X 1Q ,U Q ,Q\Y). 

Defining U = (Uq, Q) and X\q = X\ then completes the proof. The existence of the reconstruction function g 2 
follows from the definition of U. The Markov Chains U — (X, Y) — Z and Z — (U, X, Y) — Xi required to factor 
the probability distribution stated in the Theorem also follow from definitions of U and X\. ■ 
We now extend Theorem Q] to the Triangular Source coding setting. 

B. Triangular Source Coding 

Theorem 2 (Rate Distortion Region for Triangular Source Coding): lZ(Di, D 2 ) for the Triangular source coding 
setting defined in section [TT] is given by the set of all rate tuples R 2 , R^) satisfying 

#i > I(X; X 1 , U\Y), 
R 2 > I(X,Y;U\Z), 
R 3 >I{X,Y;V\U,Z) 

for some p(x, y, z, u, v,xx) = p(x)p(y\x)p(z\y)p(u\x, y)p(x\ \x, y, u)p(v\x, y, u) and function g 2 : U x V x Z — >• X 2 
such that 

Edj(X,Xj) < Dj, j=l,2. 

The cardinalities for the auxiliary random variables can be upper bounded by \U\ < \X\\y\+A and |V| < (jA"!!^! + 

If Z = 0, this region reduces to the Triangular source coding region given in J4). 

The proof of the Triangular case follows that of the Cascade case, with the additional step of Node generating 
an additional description V n that is intended for Node 2. This description is then binned to reduce the rate, with 
the side information at Node 2 being U n and Z n . Node 2 first decodes U n and then V n . 
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Sketch of Achievability 

The Achievability proof is an extension of that in Theorem Q] The additional step we have here is that we 
generate 2 n ( I ( V ' X ' Y \ u ' >+e ' > V n sequences according to J\™ =1 p(vi\ui) for each u n sequence, and bin these sequences 
to 2"( / ( v '- Y < r l^ z )+ 2£ ) bins, B 3 (m 3 ), m 3 6 [1 : 2 nR3 ]. To send from Node to Node 2, Node first finds a v n 
sequence that is jointly typical with (u n ,x n ,y n ). This operation succeeds with high probability since we have 
2n(i(V;X,Y\u)+e) y n se q Uences We then send out m 3 , the bin number for v n . At Node 2, from the probability 
distribution, we have the Markov Chain (V, U) — (X, Y) — Z. Hence, the sequences are jointly typical with high 
probability. Node 2 reconstructs by looking for unique v n € B 3 (m 3 ) such that (it™, v n , z n ) are jointly typical. This 
operation succeeds with high probability since the number of sequences in B 3 {m 3 ) is 2™( / ( 1/ ^l t/ )- e ). Node 2 then 
reconstructs using the function g 2 . 

Proof of Converse: The converse is proved in two parts. In the first part, we derive the required inequalities 
and in the second part, we show that the joint probability distribution can be restricted to the form stated in the 
Theorem. 

Given a (n, 2 nR \ 2 ni?2 , 2™ fl -\ D u D 2 ) code, define U t = (X i ~ 1 ,Y i ~ 1 , Z i_1 , Z? +1 ,M 2 ) and V t = (Ui,M 3 ). We 
omit proof of the R\ and R 2 inequalities since it follows the same steps as in Theorem Q] We have 

n 

i=l 
n 

nR 2 > J2 T ( X i> Y i-> U i\Zi)- 

i=l 

For R 3 , we have 

nR 3 > H(M 3 ) 

>H(M 3 \M 2 ,Z n ) 

= I(X n ,Y n ;M 3 \M 2 ,Z n ) 

n 

= H(X u Yi\M 2 , Z n ,JP-\ Y*- 1 ) - H(X f , Yi\M 2 , M 3 , Z n 1 X i ~ 1 , Y^ 1 ) 

i=l 
n 

= ^2 HiX^Yipi, Zi) - H(X U Yi\Ui, Vt, Zi) 

i=l 
n 

= Y,I{X i ,Y i ;V i \U i ,Z i ). 
i=i 

Next, let Q be a random variable uniformly distributed over [1 : n] and independent of (X n ,Y n , Z n ). Defining 
U = (Uq,Q), V = {Vq,Q) and X\q = X\ then gives us the bounds stated in Theorem[2] The existence of the 
reconstruction function g 2 follows from the definition of U and V. Next, from the definitions of U, V and X±, 
we note the following Markov relation: (U, V, Xi) — (X, Y) — Z. The joint probability distribution can then be 
factored as p(x, y, z, u, v, x\) = p(x, y, z)p(u\x, y)p{x\, v\x, y, u). 

We now show that it suffices to restrict the joint probability distributions to the form 
p(x, y, z)p(u\x, y)p{x\\x, y, u)p(v\x, y, u) using a method in J4] Lemma 5]. The basic idea is that since the inequali- 
ties derived rely on p(xi,v\x, y, u) only through the marginals p(x\ \x, y, u) and p(v\x, y, u), we can obtain the same 
bounds even when the probability distribution is restricted to the form p(x,y, z)p(u\x,y)p(xi\x,y,u)p(v\x,y,u). 

Fix a joint distribution p(x,y, z)p(u\x,y)p(xi,v\x,y,u) and let p(v\x, y, u) and p(x\ \x, y, u) be the induced 
conditional distributions. Note that p(x, y, z)p(u\x, y)p(x\, v\x, y, u) and p(x, y, z)p(u\x, y)p{x\ \x, y, u)p(v\x, y, u) 
have the same marginals p(x, y, z, u, v) and p{x, y, z, u, xi), and the Markov condition (U, V,X\) — (X, Y) — Z 
continues to hold under p{x 1 y, z)p(u\x, y)p(x\ \x, y, u)p(v\x, y, u). 

Finally, note that the rate and distortion constraints given in Theorem [2] depends on the joint distribution only 
through the marginals p(x, y, z, u, v) and p(x, y, z, u, x\). It therefore suffices to restrict the probability distributions 
to the form p(x, y, z)p(u\x, y)p(x\ \x, y, u)p(v\x, y,u). ■ 
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C. Two Way Cascade Source Coding 

We now extend the source coding settings to include the case where Node requires a lossy version of Z. 
We first consider the Two Way Cascade Source coding setting defined in section [TT] (we will use R3 to denote 
the rate on the link from Node 2 to Node 0). In the forward part, the achievable scheme consists of using the 
achievable scheme for the Cascade source coding case. Node 2 then sends back a description of Z n to Node 0, 
with X n , Y n , U{ 1 as side information at Node 0. For the converse, we rely on the techniques introduced and also 
on a technique for establishing Markovity of random variables found in |5). 

Theorem 3 (Rate Distortion Region for Two Way Cascade Source Coding): 1Z(Di, D 2 , D3) for Two Way Cas- 
cade Source Coding is given by the set of all rate tuples (R\, R2, R3) satisfying 

R 1 >I{X;X li U 1 \Y), 

R 2 >I(U i; X,Y\Z), 

Ra^I^Zp^X^), 

for some p(x, y, z, u\, u 2 , Xi) = p{x)p{y\x)p{z\y)p{ui \x, y)p{x\ \u\, x, y)p{u2\z, u\) and functions 172 : U\ X Z — >■ 
X 2 and g 3 : U\ x U2 X X x y — > Z such that 

EidjiX^^KDj, j = 1,2 
E(d 3 (Z,Z))<D 3 . 

The cardinalities for the auxiliary random variables can be upper bounded by \Ui\ < \X\\y\ + 5 and \U2\ < 

If Y = X, this region reduces to the result for two way source coding found in 0. 
Sketch of Achiev ability 

The forward path (R\ and R2) follows from the Cascade source coding case in Theorem[T] The reverse direction 
follows by the following. For each u™, we generate 2"( / C 72;Z l £/l )+ e ) u 1 ^ sequences according to IlILi p( u 2i\uu) 
and bin them to 2 n(I(Ur ' Z ^ Ul ' X ' Y '> +2 ^ bins, #3(7713), m 3 G [1 : 2 nRs ]. Node 2 finds a sequence that is jointly 
typical with (u",z n ). Since there are 2™( / C 72;Z l £/l )+ e ) sequences, this operation succeeds with high probability. It 
then sends out the bin index m 3 , which the jointly typical v n sequence is in. At Node 0, it recovers u 2 l by looking 
for the unique sequence in S 3 (m 3 ) such that (u™, ttj , x n , y n ) are jointly typical. From the Markov condition 
U2 — (U\, Z) — (X, Y) and the Markov Lemma (g|, the sequences are jointly typical with high probability. Next, 
since there are only 2 n ^ I ^ U2 ' ,x,Y ^ Ul ^~^ sequences in the bin, the probability that we do not find the unique (correct) 
sequence goes to zero with n. Finally, Node reconstructs using the function g 3 . 

Proof of Converse: Given a (n, 2 nRl , 2 ni?2 , 2 nRs , D u D 2 , As) code, define U u = (M 2 , X 1 " 1 , F'" 1 , Z™ +1 ) 
and U 2i = M 3 . We have 

nRi > H(Mi) 

> H(M 1 \Y n ,Z n ) 

= H(M 1 ,M 2 \Y n ,Z n ) 

= I(X n ;M 1 ,M 2 \Y n ,Z n ) 

n 

= Y,I(Xi\M 1 ,M 2 \X i -\Y n ,Z n ) 

i=l 
n 

= H^X*- 1 , Y n , Z n ) - HiXilX*- 1 , Y n , Z n , M X ,M 2 ) 

i=l 
n 

= E(Xi\Yi, Zi) - H(X i \X i ~ 1 , Y n , Z n ,M 1 ,M 2 ) 

i=l 
n 

( = ] Y, HiXilYi) - H(X i \X i ~ 1 , Y n , Z n ,M 1 ,M 2 ) 

i=l 
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y J2 HiX^Yi) - H{X i \X i - 1 ,X lh Y n , Z n , Mi, M 2 ) 

i=l 
n 

^^HiXilYt) - HiXilXn^Uu) 

i=l 
n 

= Y,I{X i ;X li ,U li \Y i ), 

i=l 

where step (a) follows from the Markov assumption X^ — Yi — Zi and step (b) follows from Xu being a function 

of (y™,Mi). 

Consider now R 2 

nR 2 = H(M 2 ) 

> H(M 2 \Z n ) 

= I(M 2 ;X n ,Y n \Z n ) 

n 

= Y, B(X t ,Y t \Z n ,X l ~ x ,Y l ~ v ) - H(X h Y,\Z n , X l -\Y l -\M 2 ) 

i=l 
n 

^Y^IiXi^UulZi). 

i=l 

Next, consider R 3 

nR 3 = H(M 3 ) 

> H(M 3 \X n ,Y") 

> I{M 3 ]Z n \X n ,Y n ) 

= H(Z n \X n ,Y n ) - H(Z n \X n , Y n ,M 3 ) 
= H{Z n \X n ,Y n ) - H(Z n \X n ,Y n ,M 2 ,M 3 ) 

n 

>Y / H(Z t \X l ,Y l )-H(Z l \Z? +1 ,X\Y\M 2 ,M 3 ) 

i=l 
n 

= Y,i{Z i ;U lh u 2i \x i ,Y i ) 

i=l 
n 

= J2 I (z i ;U2i\x i ,Y i ,u u ), 

1=1 

where the last step follows from the Markov relation Z± — (X;, Yi) — Uu which we will now prove, together with 
other Markov relations between the random variables. The first two Markov relations below are used for factoring 
the joint probability distribution while Markov relations three and four are used for establishing the distortion 
constraints. We will use the following lemma from Q. 

Lemma 1: Let A\ , A 2 , B\ , B 2 be random variables with joint probability mass functions mf p(ai,a 2 ,bi,b 2 ) = 
p(ai,bi)p(a 2 ,b 2 ). Let Mi be a function of (Ai,A 2 ) and M 2 be a function of (Bi, B 2 , Mi). Then, 

I(A 2 ;B 1 \M 1 ,M 2 ,A 1 ,B 2 )=0, (1) 
I(B 1 ;M 1 \A 1 ,B 2 )=0, (2) 
I{A 2 ;M 2 \M 1 ,A 1 ,B 2 )=0. (3) 

Now, let us show the following Markov relations: 
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1) Z, — (Xi, Yi) — (Uu, X\i): To establish this relation, we show that T(Zi\ Uu, Xij|X;, F) = 0. 

I(Zi]X u , Uu\Xi, Y.^ = I(Z i - 1 X u , M 2 , X % ~ x , Y l ~ x , Z™ +1 \Xi,Yi) 

< I(Z i ;Xi i ,M 2 ,X t ,Y l 1 , X™|_ 1 ,y i ™ 1 , Zl l +1 \Xi,Yi) 

j(7 . yi— 1 vi— 1 v n V n 7 n I y v\ 

= 0. 

2) t/ 2i - (2i, E/ii) - F): Note that U 2l = M 3 . Consider 

/(X, X, F; E/ 2l |Z,, U u ) < I(X it X?, Y*;M 3 \Z?, X l ~\ Y*~\ M 2 ) 
= /(XJ\ Y^;M 3 \Z^,X i - 1 ,Y i -\M 2 ). 

Now, using Lemma 1, set A 1 = (X^F*" 1 ), B 1 = Z*" 1 , A 2 = (Xf,^™), B 2 = (Zf), M 2 = M 3 and 
Mi = A/ 2 . Then, using the third expression in the Lemma, we see that 7(X™, F/ 1 ; M 3 |Z™, X i_1 , F i_1 , M 2 ) = 
0. 

3) Z*- 1 - (C/ H , Z,-) - (X, F): Consider 

/(X, Y; Z^lUu, < I{X?,Y?\ Z i_1 |X l_1 , F i_1 , Z™, M 2 ) 

= ff(Z i - 1 |X i - 1 ,F i - 1 ,Zf,M 2 ) - ff(Z i - 1 |X n ,F",Zf,M 2 ) 
< ff(Z i - 1 |X''- 1 ,F i - 1 ,Zf)- J ff(Z i - 1 |X n ,F n ,Zf) 
= H(Z''- 1 \X l -\Y 1 - 1 ) - H{Z i - 1 \X i ~ 1 , Y 1 - 1 ) 
= 0. 

4) (Xf +l , Y&j) - (Uu, U 2i , X, F) - Z,: Consider 

7(Xf +1 , F^; Z 4 |C/ H , f/ 2i , X, F) < 7(Xf +1 , F 4 ™ l5 Z'|M 2 , M 3 , Zf +1 , X\ F*). 

Applying the first expression in the Lemma with A 2 = (XS_ 1; Y^), Ai = (X l ,Y l ), i?! = Z l and B 2 = 
Z? +1 gives 7(Xf +1) F^; Zi\U u , U 2i , X, F) = 0. 
Distortion constraints 

We show that the auxiliary definitions satisfy the distortion constraints by showing the existence of functions 

x^(U u ,Zi) and z*(U u ,U 2i , X u Yi) such that 

E(d 2 (X i ,x* 2l (U u ,Z l ))) < E(rf 2 (X,i 2j (M 2 ,Z"))) 

E(d 2 (Z i ,z*(U li ,U2i,X i ,Y))) < E(d 2 (X i ,z 3i (M 3 ,X n ,Y n ,Z n ))), 

where x 2 i(M 2 , Z n ) and ii(M 3 ,X n ,Y n ) are the original reconstruction functions. 
To prove the first expression, we have 

E(d 2 (X, x 2i (M 2 , Z n ))) = Y,P(x\y\ «"> m 2 )d 2 {xi,x 2i (m 2 , z n )) 

— ^P(uii, z l )p(x l ,y i \uu, z l )d 2 (x l ,x' 2i (ui l ,z l , z 1 ' 1 )) 

= ^p(uu,Zi, z t ~ 1 )p(x i ,y i \uu, Zi)d 2 (xi,x' 2i (uu, z i} z 1 ' 1 )), 

where (a) follows from defining x' 2i (uu, Zi, z 1 ^ 1 ) = £ 2 i(m 2 , z n ) for all x 1-1 , y 1 ^ 1 and the last step follows from 
the Markov relation Z* -1 - (Uu, Zi) - (X,, F). Finally, defining 

(2-1-1)* = argmin^.-i £\ . y - p(xi,yi\u u , Zi)d 2 (xi,x' 2i (uu, Zi, z 1 ^ 1 )) and x* 2l (uu, z t ) = x' 2i (u u , z%, (z 1 " 1 )*) shows 
that E(d 2 (X l ,x* 2t (U ll ,Z l )jj < E(d 2 (X,i 2l (M 2 ,Z™))) as required. 

To prove the second expression, we follow similar steps. Considering the expected distortion, we have 

E(d s (Z i ,z i (M 3 ,X n ,Y n ))) 

= 7^P{z7, x n ,y n ,m 3 )d 3 (zi, z l (m 3l x n , y n )) 
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= ^K u i*J u 2i>£i,yi)Ei+i)2/^i)p(zt|«^ 

= ^p{uii,U2i,Xi,yi,x™ +1 ,y" +1 )p(z l \u ll ,U2i,x l ,y l )d 3 (z i Jt(u 1 i,U2i,x l ,y l ,x^ 

where the last step uses Markov relation 4. The rest of the proof is omitted since it uses the same steps as the 
proof for the first distortion constraint. 

Finally, using the standard time sharing random variable Q as before and defining U\ = (U\q,Q), U2 = U2Q, 
X\ = Xiq, we obtain the required outer bound for the rate-distortion region. ■ 

We now turn to the final case of Two Way Triangular Source Coding. 



D. Two Way Triangular Source Coding 

Theorem 4 (Rate Distortion Region for Two Way Triangular Source Coding): lZ(Di, D 2 , D 3 ) for Two Way Tri- 
angular Source Coding is given by the set of all rate tuples R 2 , R 3 , R4) satisfying 

Rt^IiX-XuUtlY), (4) 

R 2 >I(X,Y;U 1 \Z), (5) 

R 3 >I{X,Y;V\Z,U X ), (6) 

R 4 >I(U 2 ;Z\U 1 ,V,X,Y), (7) 

for some p(x,y, z,ui,u 2 ,v,x 1 ) = p(x)p(y\x)p(z\y)p(ui\x,y)p(x 1 \x,y,ui)p(v\x,y,u 1 )p(u 2 \z,ui,v) and func- 
tions g 2 : U\ x V x Z — s- X 2 and g 3 :U\ xU 2 xV x X x y ^ Z such that 

E(di(X, Xi)) < D±, (8) 
£{d 2 {X,X 2 )) <D 2l (9) 
E(d 3 {Z,Z))<D 3 . (10) 

The cardinalities for the auxiliary random variables are upper bounded by \Ui\ < \X\\y\+6, |V| < |Wi | (| |3^| + 3) 
and \U 2 \ < |Wi||V|(|^| + l). 
Sketch of Achiev ability 

The forward direction (R\, R 2 , R3) for Two-Way triangular source coding follows the procedure in Theorem|2] 
For the reverse direction (R4), it follows Theorem [3] with (Ui,V) replacing the role of U\ in Theorem [5] 

Proof of Converse: Given a (n, 2 nRl , T lRi , 2™ fla , 2 nRi , D u D 2 , D 3 ) code, define U u = (M 2 , X 1 - 1 , Y*' 1 , Z? +l ), 
U 2 i = M4 and Vi = (M 3 , Uu). The R\ and R 2 bounds follow the same steps as in Theorem 3. For R 3 , we have 

nR 3 > H(M 3 ) 

>H(M 3 \M 2 ,Z n ) 

= I(X n ,Y n ;M 3 \M 2 ,Z n ) 

n 

= H(X h Yi\M 2 , Z n 1 X l ~\Y 1 - 1 ) - H(Xi, Y t \M 2i M 3 , Z n , X i_1 , 

1=1 
7 1 

> H(X U Yi\U u Zi) - H{Xi,Yi\Uu, V t , Z t ) 

i=l 
n 

= J2l(X i ,Y i ;V i \U u ,Z i ). 



Next, consider 



ni? 4 = ff(M 4 ) 

> H(M 4 \X n ,Y n ) 

> I(M 4 ;Z n \X n ,Y n ) 

= H(Z n \X n , Y n ) - H{Z n \X n , Y n , M 4 ) 
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= H(Z n \X n , Y n ) - H(Z n \X n , Y n , M 2 , M 3 ,M 4 ) 

n 

>J2 H ( z i\X i ,Y i )-H{Z i \Z^ 1> X i > Y i ,M 2 ,M 3> M 4 ) 

i=l 
n 

= J2 I (z i ;U u ,v i ,u 2i \x i ,Y i ) 

i=l 
n 

= Y / I ( z ^U2i\X i ,Y i ,Vi,U u ), 

i=l 

where the last step follows from the Markov relation Z% — (Xi , Yi) — {Vi, Uu) which we will now prove together with 
other Markov relations between the random variables. The first 2 Markov relations are for factoring the probability 
distribution while Markov relations 3 and 4 are for establishing the distortion constraints. 
Markov Relations 

1) Zi - (Xi,Yi) - (U u , Vi, Xu): To establish this relation, we show that I{Zi;X u , U u , V\X l ,Y i ) = 0. 

I(Z f , X u , U u , Vi\X h Yi) = I(Zi-,X u , M 3 , M 2 , X i_1 ,Y i_1 , Z? +1 \X i: Y) 

< I[Zi\ X u , M 3 ,M 2 , X'-^Y'-^X^Y^Z^Xi, Y z ) 

T(7 . yi—1 vi~l yn yn 711 \v \r\ 

— l\Zji,A , I 

= 0. 

2) U 2l - {Zi, U u , Vi) - {X u , Xi,Yi): Consider 

I{X h X u Y t ; U 2i \Z h U u , V.i) < I(Xi,X?, Y t n ; M 4 \Z? , X 1 " 1 , F 1 " 1 ,M 2 ,M 3 ) 

= 7(Xf , F"; M A \Z?, X 1 - 1 , Y l ~\ M 2 , M 3 ). 

Now, using Lemma 1, set A 1 = (X*- 1 , Y*" 1 ), B 1 = Z*" 1 , A 2 = (X?,Yf), B 2 = {Zf), M 2 = M 4 and 
Mi = M 2 . Then, using the third expression in the Lemma, we see that I{Xf , Yf; M 4 \Zf, Y i_1 , M 2 ) = 

0. 

3) Z*- 1 - {Uu, V h Zi) - {Xi,Yi): Consider 

I{Xi,Yi\ Z^U^Vi, Zi) < I{X?,Yf; Z i ~ 1 \X l ~ 1 ,Y i ~ 1 , Z™, M 2 , M 3 ) 

= H{Z i - 1 \X i - 1 ,Y i ~ 1 ,Zf,M 2 ,M 3 ) - H{Z i - 1 \X n ,Y n ,Zl l ,M 2 ,M 3 ) 
< H{Z i - l \X i - 1 ,Y i ~ l ,Zf) -H{Z i ~ l \X n ,Y n ,Zf) 
= H{Z l - 1 \X l -\Y 1 - 1 ) - H{Z l - 1 \X l -\Y 1 - 1 ) 
= 0. 

4) (Xf +1 , Yfrj) - {Uu, U 2i , Vi,Xi,Yi) - Z. r . Consider 

I{X^ +1 ,Y^ +1 ; Z t \Uu, U 2i , V h Xi,Yi) < I{X" +1 , Y^ ; Z i \ M 2 , M 3 , M 4 , Z? +1 ,X\ Y*). 

Applying the first expression in the Lemma with A 2 = {XV\ r , Y-\ -J, A\ = {X l ,Y % ), B\ = Z l and B 2 = 
Z? +1 gives I{X? +1 ,Y? +1 ;Zi\U u , U 2i , X h Yi) = 0. 
Distortion Constraints 

The proof of the distortion constraints is omitted since it follows similar steps to the Two Way Cascade Source 
Coding case, with the new Markov relations 3 and 4, and {Uu, Vi) replacing Uu in the proof. 

Using the standard time sharing random variable Q as before and defining U\ = {U\q, Q), U 2 = U 2 q, X\ = X\q 
and V = Vq we obtain an outer bound for the rate-distortion region for some probability distribution of the 
form p{x, y, z, u\, u 2 , v, x\) = p{x, y, z)p{u\\x, y)p{x\, v\x, y, ui)p{u 2 \z , u\, v). It remains to show that it suffices 
to consider probability distributions of the form p{x, y, z)p(ui\x, y)p{xi\x,y, Ui)p{v\x, y, Ui)p{u 2 \z, m, v). This 
follows similar steps to proof of Theorem 2. Let 

Pi = P{x, y, z)p{u x \x, y)p{x i, v\x, y, ux)p{u 2 \z, Ui,v), 

V2 = p{x, y, z)p{u x \x, y)p{xi \x, y, m)p{v\x, y, m)p{u 2 \z, u x ,v), 
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where p(xi\x, y, Ui) and p(v\x, y, u\) are the marginals induced by p\. Next, note that Ri, R2, R3, R4 and the 
distortion constraints depend on p\ only through the marginals p(x, y, z, u\, u 2 , v) and p(x, y, z, ui,xi). Since these 
marginals are the same forpi andp 2 , the rate and distortion constraints are unchanged. Finally, note that the Markov 
relations 1 and 2 implied by pi continues to hold under p2- This completes the proof of the converse. ■ 

IV. Gaussian Quadratic Distortion Case 

In this section, we evaluate the rate-distortion regions when (X, Y, Z) are jointly Gaussian and the distortion is 
measured in terms of the mean square error. We will assume, without loss of generality, that X = A + B + Z, 
Y = B + Z and Z = Z, where A, B and Z are independent, zero mean Gaussian random variables with variances 
a\, a% and a\ respectively. 

A. Quadratic Gaussian Cascade Source Coding 

1 CT 2 +CT 2 

Corollary 1 (Quadratic Gaussian Cascade Source Coding): First, we note that if R2 < ^ log A D . B , then the 

2 , 2 2 

distortion constraint D 2 cannot be met. Hence, given D U D 2 > and R 2 > max{± log Za ^ s -, 0}, the rate 
distortion region for Quadratic Gaussian Cascade Source Coding is characterized by the smallest rate i?i such that 
(Di,D 2 ,R x ,R 2 ) are achievable, which is 

fl. 1. o\ \ 
Ri = max <^ - log — , - log — \ , 

[ 2 n i 1 g a\u,b J 

where U = a* A + {3*B + Z* , Z* ~ A^(0, cr%*), with a*, (3* and a\, achieving the maximum in the following 
optimization problem: 

2 



maximize Qa\ub 



subject to i?2 > ^ log 



D 2 > v 2 a+b\u 

The optimization problem given in the corollary can be solved following analysis in |4). In our proof of the corollary, 
we will show that the rate distortion region obtained is the same as the case when the degraded side information 
Z is available to all nodes. 

Converse: Consider the case when the side information Z is available to all nodes. Without loss of generality, 
we can subtract the side information away from X and Y to obtain a rate distortion problem involving only A + B 
and B at Node 0, B at Node 1 and no side information at Node 2. Characterization of this class of Quadratic 
Gaussian Cascade source coding problem has been carried out in |4) and following the analysis therein, we can 
show that the rate distortion region is given by the region in Corollary Q] ■ 

Achiev ability: We evaluate Theorem [TJ using Gaussian auxiliaries random variables. Let U' = a*X + ((3* — 
a*)Y + Z* = a* A + (3* (B + Z) + Z* and V be a Gaussian random variable that we will specify in the proof. 
We now rewrite Ri = I(X; U', Xl|Y) as R 1 = I(X; U', V\Y) with X X =V+ E{X\U' , Y), V independent of U' 
and Y. Let g 2 (U', Z) = E(X\U', Z). Evaluating R\ and R2 using this choice of auxiliaries, we have 

Ri = I(X\U',V\Y) 

= h(A + B + Z\B + Z) - h(X\U', V, Y) 

1 , °\ 
= - lor 



9 6 cr 2 , 

U X\U',V,Y 

R 2 = I(X,Y;U'\Z) 

= h{U'\Z) - h(U'\X,Y,Z) 
1 



los 

2 S 



T u*A+P»B+Z* 



•V 
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Next, we have 



2 2 
a X\U',Y = a A+B+Z\a*A+p*(B+Z) + Z*,B+Z 
_ 2 

— "A\a*A+Z',B+Z 

— J 1 

— "A\u*A+Z* 
_ _2 

— "A\U,B- 



V = X - E(X\U',Y) + Z 2 where Z 2 ~ JV(0, Dia x , vl Y /( a x\U' y ~ ^i)) so that CT i|c/' ky = Dl and obtain 



^ a x\U' y = a A\UB — we set V — to obtain R\ — \ log ai tTA — . If <Jx\u' y > then we choose 
V = X - E(X\U',Y) + Z 2 where Z 2 ~ A^(0, T)\a^^j, y / (c^-iy, ^ - £>i)) so that o-| |E/ , 

2 2 " 

i?i = | log Therefore, i?i = max{i log J 1 , | log 
ow that 



5 /■ 

Finally, we show that this choice of random variables satisfy the distortion constraints. For D\, note that since 
E(X — Xi) 2 = <Jx\u' vy> me distortion constraint D\ is always satisfied. For the second distortion constraint, we 
have 



E(X - X 2 f = a x 



2 

X\U'Z 



2 

"A+B\a*A+p*(B+Z) + Z>,Z 
2 

a A+B\a* A+P*B+Z» ,Z 
2 

° A+B\a* A+/3* B + Z' 
2 

a A+B\U 



< D 2 . 

Hence, our choice of auxiliary U' and V satisfies the rate distortion region and distortion constraints given in 
the corollary, which completes our proof. ■ 



B. Quadratic Gaussian Triangular Source Coding 

Corollary 2 (Quadratic Gaussian Triangular Source Coding): Given D\ 1 D 2 > and R 2l R% > 0, R 2 + R3 > 
I log - A ^- B -, the rate distortion region for Quadratic Gaussian Triangular Source Coding is characterized by the 
smallest Ri for which (£>i, D 2 , Ri, R 2 , R3) is achievable, which is 




i?i = max ■ 

where U = a* A + j3* B + Z* , Z ~ N(0,a z ,), with a*, ft* and a 2 z , satisfying the following optimization problem. 



maximize o~\\u b 

subject to R 2 >\- log ^j- 

2 o z „ 

2 2R3 D 2 > a 2 A+B{u 

As with Corollary [U the optimization problem given this corollary can be solved following analysis in H. 
Converse: The converse uses the same approach as Corollary [T] Consider the case when the side information 
Z is available to all nodes. Without loss of generality, we can subtract the side information away from X and Y 
to obtain a rate distortion problem involving only A + B and B at Node 0, B at Node 1 and no side information 
at Node 2. Characterization of this class of Quadratic Gaussian Triangular source coding problem has been carried 
out in JH and following the analysis therein, we can show that the rate distortion region is given by the region in 
Corollary |2] ■ 
Achiev ability: We evaluate Theorem |2] using Gaussian auxiliary random variables. Let U' = a*X + (Jj* — 
a*)Y + Z* = a* A + f3*{B + Z) + Z* and V = X + r]U' + Z 3 , Z 3 ~ N(0, u% ). Following the analysis in 
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Corollary Q] the inequalities for the rates are 



i?i = max < - log , - lo 



2 ~° D 1 ' 2 " ° cr 2 



A\U,B 

1 

''Z> 

R 3 > I(X 7 Y- V\Z, U') = I(X; V'\Z, U') 



Z "X\Z,U',V 

As with Corollary Q] the distortion constraint D\ is satisfied with an appropriate choice of X\. For the distortion 
constraint D2, we have 



D 2 > a 



2 

X\Z,U' ,v- 



Next, note that we can assume equality for i?3, since we can adjust r\ and a z so that inequality is met. Since 
this operation can will only decrease cr x \zu' V" me distortion constraint D2 will still be met. Therefore, setting 

R 3 = 5 log f x,z ' u ' , we have 



XIZ.U' ,v 



D2 > Vx\Z,U',V> 
2 

(T X\Z,U' 



2 2R 3 ■ 

Since <? 2 X \ Z y, = Va+biu' ^ s completes the proof of achievability. ■ 
Remark: As alternative characterizations, we show in Appendix [C] that the Cascade and Triangular settings in 
Corollaries [T] and |2] can be transformed into equivalent problems in |4J where explicit characterizations of the rate 
distortion regions were given. 



C. Quadratic Gaussian Two Way Source Coding 

It is straightforward to extend Corollaries[T]and|2]to Quadratic Gaussian Two Way Cascade and Triangular Source 

Coding using the observation that in the Quadratic Gaussian case, side information at the encoder does not reduce 

2 

the required rate. Therefore, the backward rate from Node 2 to Node is always lower bounded by \ log z ^ +z . 
This rate (and distortion constraint D 3 ) can be achieved by simply encoding Z. We therefore state the following 
corollary without proof. 

Corollary 3 (Quadratic Gaussian Two Way Triangular Source Coding): Given D\, D2, -D3 > 0, i?2,^?3 > 0, R 2 + 

R3 > \ log " a d" b anc J ^ 4 — max {^ log 1 0}. the rate distortion region for Quadratic Gaussian Two Way 
Triangular Source Coding is characterized by the smallest R\ for which (R±, R2, R3, R4, D\, D2, D3) is achievable, 
which is 

fl. oil, u\ \ 
R x = max <^ - log — , - log — \ , 

{ 2 v i 1 °a\u,b J 

where U = a* A + /3* B + Z* , Z ~ N(Q,o~\,), with a*, ft* and a 2 z , satisfying the following optimization problem. 

U,B 

i- 
2 

' A+B\U 

Remark: The special case of Two Way Cascade Quadratic Gaussian Source Coding can be obtained as a special 



maximize o~a\ub 



subject to i?2 > 2 ~1T~ 



''D 2 > ai 



case by setting R3 = 0. 
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Next, we present an extension to our settings for which we can characterize the rate-distortion region in the 
Quadratic Gaussian case. In this extended setting, we have Cascade setting from Node to Node 2 and a triangular 
setting from Node 2 to Node 0, with the additional constraint that Node 1 also reconstructs a lossy version of Z. 
As formal definitions are natural extensions of those presented in section [II] we will omit them here. The setting 
is shown in Figure [5] 




Node 



Fig. 5: Extended Quadratic Gaussian Two Way source coding 
Theorem 5 (Extended Quadratic Gaussian Two Way Cascade Source Coding): Given Di,D2 > 0, < D Zl ,Dz 2 < 

2,2 

cr z , Y and i?2 > max{| log ry ' A ^ B , 0}, the rate distortion region for the Extended Quadratic Gaussian Two Way 
Cascade Source Coding is given by the set of Ri, R3, R4, R$ > satisfying the following equalities and inequalities 

Ri = ma 

where U = a*A + /3*B + Z*, Z* ~ 7V(0, ct|„ ), with a*, (3* and er|» satisfying the following optimization problem. 

2 




maximize o~a\ub 



subject to i? 2 > - log ^jr- 



D 2 > 



a A+B\U 



and 



R3 > o lo S 



2 

a Z\Y 



R 3 + R 5 >^ log 



2 D Zl ' 

1 . a \\Y 



2 6 mm{D Zl ,D Z2 y 



1 , a Z\Y 



R 4 + R 5 > ^log 

Proof: 
Converse 

For the forward direction (Ri,R2), we note that Node 2 can only send a function of (Mi, Y n , Z n ) to Nodes 
and 1 using the R4 and i?5 links. Since Mi and Y n available at both Node and 1, the forward rates are lower 
bounded by the setting where Z n is available to all nodes. Further, in this setting, the distortion constraints D Zl 
and Dz 2 are automatically satisfied since Z is available at Nodes and 1. Therefore, (R3, R4, R$) do not affect 
the achievable (i?i,i?2) rates in this modified (lower bound) setting. (i?i,i?2) are then obtained by the observation 
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in Corollary Q] that the rate distortion region obtained for our Quadratic Gaussian Cascade setting in Corollary Q] is 
equivalent to the case where the side information Z is available at all nodes. 

For the reverse direction, the lower bounds are derived by letting the side information (X, Y) to be available 
at Node 2, and for side information X to be available at Node 1. The D\ and D 2 distortion constraints are then 
automatically satisfied since X is available at all nodes. We then observed that (Ri,R 2 ) do not affect the achievable 
(i?3, i?4, Rc,) rates in this modified (lower bound) setting. The stated inequalities for R 3 , R4, R 5 are then obtained 
from standard cutset bound arguments and the fact that X — Y — Z form a Markov Chain. 

Achiev ability 

We analyze only the backward rates R 3l R± and R$ since the forward direction follows from Corollary Q] For 
the backward rates, we now show that the rates are achievable without the assumption of (X, Y) being available 
at Node 2. We will rely on results on successive refinement of Gaussian sources with common side information 
given in |9). A simplified figure of the setup for analyzing the backward rates is given in Figure [6] We have three 
cases to consider. 



Node 



Node 1 




Node 2 



Fig. 6: Setup for analysis of achievability of backward rates 



Case 1: D Zl < D Z2 

In this case, the inequalities in the lower bound reduce to 

R3 > \ log 



R 4 



R 5 > 2 log 



U Z\Y 

2 

a Z\Y 



From the successive refinement results in (9), we can show that the following rates are achievable 

R 3 = I(U 1 ,U 2 ,U 3 ;Z\Y), 
Ri = I(U 2 ;Z\Y), 
R 5 =I(U 3 ;Z\Y,U 2 ) 

for some conditional distribution F(Ui, U 2 , U 3 \Z), Zi(Ui 1 U 2l U 3l Y) and Z 2 (Ui 1 U 2l Y) satisfying the distortion 
constraints. Now, for fixed R4 < \ log ^ , choose D'(> Dz 2 ) such that R4 — ^ log ^ . We now choose the 

auxiliary random variables and reconstruction functions in the following manner. Define Q(x) :— x ° z ^ m 



Ui = Z + Wx where W\ - 
U 3 = Ui + W 3 where W 3 
U 2 = U 3 + W 2 where W 2 
Z x = E(Z\U U Y), 
Z 2 = E{Z\U 3 ,Y). 



N(0,Q(D Zl )), 
^N{0,Q{D Z2 )-Q{D Zl )), 
^N(0,Q(D')-Q(D Z2 )), 
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From this choice of auxiliary random variables, it is easy to verify the following 

R 3 =I(U 1 ,U 2 ,U 3 ;Z\Y) 
= I(U i; Z\Y) 

R 4 = I(U 2 ;Z\Y) 

T Z\Y 



1, °\ 
— l02 



2 ° D' ' 

i? 4 + # 5 = /(Ik; zpO + ^ 3 ; iM 
i . 4|r 

E(Z-Zi) a = £>^, 
E(Z - Z 2 ) 2 = D 



z 2 - 



Case 2: D Zl > Dz 2 , R 3 > Ra 

In this case, the active inequalities are 



p 1 1 °V 

Zi 

1 , ®Z\Y 



2 D 
R4 + R 5 >1 lo, 



2 ° Z?z 2 

From [9 1, the following rates are achievable 

J2 3 = 7(l7 1 ,l7 2 ;Z|y) ) 

i? 4 = /(f/ 2 ;^|y), 

i? 5 = J([/ 3j C/i;^|y,f/ 2 ). 

2 

First, assume R 3 < | log ^ . Choose Dz 2 < -D' < -D" < -Dzi- We choose the auxiliary random variables and 
reconstruction functions as follows. 

U 3 = Z + W 3 where W 3 ~ 7V(0, Q(D Za )), 

U 1 = U 3 + Wi where Wi ~ N(0, Q(D') - Q(£>z a )), 

U 2 = U 1 + W 2 where W 2 - N(0, Q(D") - Q(D')), 

Z x = E{Z\U U Y), 

Z 2 = E{Z\U 3 ,Y). 

From this choice of auxiliary random variables, it is easy to verify the following 

R 3 = I{U X ,U 2 ;Z\Y) 
= I(U i; Z\Y) 

1 , °%\Y 

= 2 l0g -D^' 
R A = I{U 2 -Z\Y) 

1 . 

= 2 l0g -D^' 
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i? 4 + R 5 = I(U 2 ; Z\Y) + I(U 3 , U X ;Z\Y, U 2 ) 
= I(U 3 ,U 1 ,U 2 ;Z\Y) 
= I(U 3 ;Z\Y) 

1 . °%\Y 

E(Z - Z x f = D' < D Zl , 
E(Z-Z 2 ) 2 = D 



Next, consider R3 > i log |y and i?4 > | log -M |y . Then, it is easy to see from our achievability scheme 

that we can obtain i?4 < i?4, 7? 3 < i?3 and i?5 = by setting D' = D" = Dz 2 - Finally, consider the case where 

2 2 

R 3 > ilog^ and R A < ilog^P-. Then, we observe from our achievability scheme that we can achieve 

A U Z 2 " U Z-y 

R' 3 = I log ^ Y < i?3 for any R4 and R$ satisfying the inequalities by setting D' = Dz 2 ■ 
Case 3: Dz 1 > Dz 2 , R3 < Ra 



In this case, the active inequalities are 



Ra > ;r lo 



1 . °Z\Y 



R 3 + Rb>^ lo 



2 D Zl ' 
1 , o%\y 



2 D Z2 

^2 

We first consider the case where R3 < | log ^ . We exhibit a scheme for which R' 4 = R 3 (< #4) and still 
satisfies the constraints. This procedure is done by letting U 2 in case 2 to be equal to U\. For _D^ 2 < D' < Dz ± , 
define the auxiliary random variables and reconstruction functions as follows. 

U 3 = Z + W 3 where W 3 ~ N(0, Q(D Za )), 

U X = U 3 + Wi where Wi ~ N(0, Q(D') - Q{Dz 2 )), 

Z x = E{Z\Ut,Y), 

Z 2 = E(Z\U 3 ,Y). 



Then, we have the following. 



R 3 = I(U 1 ;Z\Y) 

1 . °\\Y 

= 2 l0g ^' 
J?4 = I(Ui; Z\Y) 

1 . °%\Y 

= 2 l0g -D^' 
R 3 + R 5 = I(U i; Z\Y) + I(U 3 ; Z\Y, U 1 ) 

= I(U 3) U 1 ;Z\Y) 

= I(U 3 ;Z\Y) 

1 . a2 z\Y 
= - log — — , 

2 S D Z2 

E(Z-Z l f = D' <D Zl , 
£{Z-Z 2 f = Dz 2 . 



19 



Finally, we note that in the case where R3 > | log -jj^-, we can always achieve R^ = | log ^ , R' A = 

■2 22 

\ log ^ and R' 5 = by letting D' = D Z2 . ■ 
Remark 1: The Two-way Cascade source coding setup given in section HH can be obtained as a special case by 
setting i? 3 = R 4 = and D Zl — > 00. 

Remark 2: The rate distortion region is the same regardless of whether Node 2 sends first, or Node sends first. 
This observation follows from (i) our result in Corollary Q] where we showed that the rate distortion region for the 
Cascade setup is equivalent to the setup where all nodes have the degraded side information Z; and (ii) our proof 
above where we showed that the backward rates are the same as in the case where the side information (X, Y) is 
available at all nodes. 

Remark 3: For arbitrary sources and distortions, the problem is open in general. Even in the Gaussian case, 
the problem is open without the Markov Chain X — Y — Z. One may also consider the setting where there is a 
triangular source coding setup in the forward path from Node to Node 2. This setting is still open, since the trade 
off in sending from Node to Node 2 and then to Node 1 versus sending directly to Node 1 from Node is not 
clear. 

V. Triangular Source Coding with a helper 

We present an extension to our Triangular source coding setup by also allowing the side information Y to be 
observed at the second node through a rate limited link (or helper). The setup is shown in Figure [7] As the formal 
definitions are natural extensions of those given in section [U] we will omit them here. 



t 




Helper 



Fig. 7: Triangular Source Coding with a helper 

Theorem 6: The rate distortion region for Triangular source coding with a helper is given by the set of rate tuples 

Rt >I{X;X u U x \Y,U h ), 
R 2 >I(U i; X,Y\Z,U h ), 
R 3 >I(X,Y;U 2 \U l7 U h ,Z), 
R h >I(U h ;Y\Z). 

for some p(x, y, z, u t , u 2 , u hl x x ) = p(x)p{y\x)p{z\y)p(u h \y)p(u\x,y,u h )p(xi\x,y,ui,u h )p(u 2 \x,y,u 1 ,u h ) and 
function g-x : U \ x U2 x Uu x Z — > X 2 such that 

EdjiX^XjKDj, j=l,2. 

We give a proof of the converse. As the achievability techniques used form a straightforward extension of the 
techniques described in Appendix lAl we give only a sketch of achievability. 
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Converse: Given a (n, 2 nRl , 2 nR2 , 2 nR3 , 2 nRh , D u D 2 ) code, define U hi = (Y i ~ 1 ,Z i ~ 1 , Z™ +1 ,Mh), U u = 
(X 1 ^ 1 , M 2 ) and U-a = (Uhi, Uu, M3). Observe that we have the required Markov conditions (Xi, Z,\) — Yi — Uhi 
and Zi — (Xi,Yi, Uhi) — {Uu, U 2 i). For the helper condition, we have 

nR h >I(M h ;Y n \Z n ) 

n 

= J2H(Y l \Z. l )~H(Y l \Y*-\M h ,Z n ) 

1=1 
71 

= ^2l(U M ;Yi\Zi). 
i=l 

For the other rates, we have 

nR\ > H(Mx) 

> H{M 1 \Y n ,Z n ) 

= H(M U M 2 \Y n , Z n ) = I(X n ; M x ,M 2 \Y n , Z n ) 

n 

= Y J I{X l ;M l ,M 2 \X l -\Y n , Z n ) 

i=l 
n 

= H^X*- 1 ^,^) - H(X h Y\X l -\Y n , Z n , Mi, M 2 ) 

i=l 
n 

= H{Xi\Y h Zi) - H(X h Y^" 1 , Y", Z n , M U M 2 ) 

n 

( = 5 Y, H ( X i\ Y i) -H(X i ,Y i \X i - 1 ,Y n ,X u ,Z n ,M 1 ,M 2 ,M h ) 

1=1 

n 

> H{Xi\Y h Um) - H{Xi\X lu Y h U u , U M ) 
i=i 

n 

= J2 I ( x i' X H,U u \Y h U hi ). 

i=l 

(a) follows from the Markov chain condition. Next, 

nR 2 > H(M 2 \M h ) 

> H(M 2 \Z n ,M h ) 

= I(X n ,Y n ;M 2 \Z n ,M h ) 



J2 1{Xi , Yi ; M 2 1 Z n , X 1 - 1 , Y l ~ 1 , M h ) 

i=l 
n 

Y H{X u Yi\Z n , X l -\Y l -\M h ) - H(X f , Y\Z n , X i ~ 1 ,Y i ~ 1 ,M 2 , M h ) 

1=1 

n 

J2 H{X u Yi\Z h Uhi) - H{X U Yi\Z u U u , U hi ) 

i=l 
n 

J2i{ x i,Yi;Uu\Zi,u hi ). 



1=1 
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Next, 



nR 3 > H(M 3 ) 

>H{M 3 \M 2 ,M h ,Z n ) 

= I{X r \Y n ] M 3 \M 2 ,M h Z n ) 

n 

= Y J H{X l ,Y^M 2 ,M h ,Z n ,X l -\Y l - 1 )-H{X l ,YMl2,M 3 ,M hl Z n ,X l -\Y 1 - 1 ) 

i=l 
n 

= J2 I (Xi,Y i ;U 2 i\U li ,U hi ,Z i ), 

i=l 

Finaly, it remains to show that the joint probability distribution induced by our choice of auxiliary random variables, 
p(x)p(y\x)p(z\y)p(uh\y)p{u\x,y,Uh)p(xi,u 2 \x,y,ui,Uh), can be decomposed into the required form. This step 
follows closely the similar step in the proof of Theorem [2] which we therefore omit. 
Sketch of Achiev ability 

The achievability follows that of Triangular source coding, with an additional step of generating a lossy description 
of Y n . The codebook generation consists of the following steps 

• Generate 2 n ( I ( Y ' Uh -" >+e ' ) U% sequences according to Y[7=iP( u hi)- Partition the set of U% sequences into 

2 n(I(U h ;Y\Z)+2e) ^ g^), mft g [1 : 2 n(I(U h ;Y\Z)+2^ 

• Generate 2 nlyI ^ x,Y,Uh ' ,Ul ' >+ ^ sequences according to Yii=iP( u n)- Partition the set of [/" sequences into 
2n(i(u 1 ;X\Y,u h )+2e) b mSi Si(niio). Separately and independently, partition the set of U n sequences into 

2 n(I(Uv,X,Y\Z,U h )+2e) ^ g^), jxi 2 G [1 :2 n(I(U;X,Y\Z)+2e)^ 

• For each (u", u^, y n ) sequence, generate 2 n ^^ Xl '< x \ u ^^ Y - u h)+(-) sequences according to Yl7=i p(%i\ u ii, u m, Hi)- 

• Generate 2™( / ( c/2;X ' F l £/h ' ,7l ) +c ) U 2 sequences according to Yii=i P( u 2i\uu, Uhi) for each (it™,itJJ) sequence, 
and partition these sequences to 2 n( - I( - U2 '< X ' Y \ Ul ' Uh ' Z '> +2 ' i '> bins, B 3 {m 3 ). 

Encoding consists of the following steps 

• Helper node: The helper node (and Nodes and 1) looks for a u 1 ^ sequence such that (u^,y n ) S Te . This 
step succeeds with high probability since there are 2 n ( I ( Y ' Uh " >+e ' > Ufi sequences. The helper then sends out the 
bin index rrih such that itJJ <G B(nih). The sequences (itJJ, x n , y n , z n ) are jointly typical with high probability 
due to the Markov Chain {X, Z) -Y - U h - 

• Node 0: Given (x n ,y n , it?) S T^ n \ Node looks for a jointly typical codeword u". This operation suc- 
ceeds with high probability since there are 2"' / ' X ' y ' C/h;C/l ' +e ) {/" sequences. Node then looks for a f n 
that is jointly typical with (u™, x n , y n , itJJ). This operation succeeds with high probability since there are 

2 n{I{X 1 -X\U 1 ,U h ,Y)+e) -n sequences 

• Node also finds a it™ sequence that is jointly typical with (it™, u%, x n , y n ). This operation succeeds with 
high probability since we have 2 n ^ It ^ U2 ' ,X ' Y ^ Ul ' Uh ^ + '^ v" sequences. 

• Node then sends out the bin index mio such that u™ S Bi(toio) and the index corresponding to x™ to Node 
1. This requires a total rate of R 1 = I(U;X\Y) + I(X?\X\U, Y) + 3e to Node 1. Node also sends out the 
bin index 7713 such that uV, £ B(m 3 ) to Node 2. This requires a rate of I(U 2 ;X, Y\Ui, Uh, Z) + 2e. 

• Node 1 decodes the codeword it" and forwards the index m 2 such that it" £ B{m 2 ) to Node 2. This requires 
a rate of I{Ur,X,Y\Z, U h ) + 2e. 

Decoding consists of the following steps 

• Node 1: Node 1 reconstructs it" by looking for the unique U" sequence in Bi(mio) such that ([/", Ufi, Y n ) £ 
Te (n) - Since there are only 2 n ^ x ' Y ' u ^' u ^-^ Ul ' x \ Y ' u ^-^ = 2 n{I{u ^ Uh ^-^ sequences in the bin, this 
operation succeeds with high probability. Node 1 reconstructs X n as X™(mio, tou). Since the sequence 
(X™, X n ) are jointly typical with high probability, the expected distortion constraint is satisfied. 

• Node 2: We note that since (U 1} U 2 , U h ,X) - Y - Z, the sequences ([/£, C/f, U$,X n ,Y n , Z n ) are jointly 
typical with high probability. Decoding at node 2 consists of the following steps 

1) Node 2 first looks for u r h l in Bhirrih) such that (u^,z n ) £ % . This operation succeeds with high 
probability since there are only 2 n ^ I ^ Uh ' Z ^~ e u r h l sequences in the bin. 
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2) It then looks for it? in B 2 {m 2 ) such that z") e 7? . Since i"(J7i; X, F, Uh)—I(Ui;X, Y\Z, Uh) = 
I{U\\ Z,Uh) by the Markov Chain Z — (X,Y,Uh) — U\, this operation succeeds with high probability 
as there are only 2 n ^ I ^ Ul ' z>Uh ' e u\ sequences in the bin. 

3) Finally, it looks for v% in B 3 (m 3 ) such that (w™, u™, u%, z n ) G 7; (n) . Since J(E/ 2 ; X, Y\U h , U x ) - 
7(C/ 2 ; X, Y\Z, U h , Ui) = I(U 2 ; Z\U r , U h ) by the Markov Chain Z - {X, Y, U h , U x ) - U 2 , this operation 
succeeds with high probability as there are only 2 n ^ I ^ U2 ' Z ^ Ul ' Uh ^~ e sequences in the bin. 

4) Node 2 then reconstructs using the function x 2 i = g 2 {un, u 2 i, Um, zi) for i S [1 : n]. Since the sequences 
(X n ,Z n ,Ui,U 2 ,UJ^) are jointly typical with high probability, the expected distortion constraint is 
satisfied. 



VI. Conclusion 

Rate distortion regions for the cascade, triangular, two-way cascade and two-way triangular source coding settings 
were established. Decoding part of the description intended for Node 2 and then re-binning it was shown to be 
optimum for our Cascade and Triangular settings. We also extended our Triangular setting to the case where there 
is an additional rate constrained helper, which observes Y, for Node 2. In the Quadratic Gaussian case, we showed 
that the auxiliary random variables can be taken to be jointly Gaussian and that the rate-distortion regions obtained 
for the Cascade and Triangular setup were equivalent to the setting where the degraded side information is available 
at all nodes. This observation allows us to transform our Cascade and Triangular settings into equivalent settings 
for which explicit characterizations are known. Characterizations of the rate distortion regions for the Quadratic 
Gaussian cases were also established in the form of tractable low dimensional optimization programs. Our Two 
Way Cascade Quadratic Gaussian setting was extended to solve a more general two way cascade scenario. The case 
of generally distributed X, Y, Z, without the degradedness assumption, remains open. 
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Appendix A 

ACHIEVABILITY PROOFS 

Achievability proof of Theorem Q] 
A. Codebook Generation 

• Fix the joint distribution p(x, y, z 7 u, ii) = p(x)p(y\x)p(z\y)p(u\x,y)p(xi\x,y,u). Let R = R w + i?u, 
Ri > R w and R 2 > R w . 

• Generate 2 nRl ° U n (l) sequences, I e [1 : 2 nRl ], each according to 117=1 

• Partition the set of U n sequences into 2 nRw bins, Bi(mio), mio G [1 : 2 nRl0 }. Separately and independently, 
partition the set of U n sequences into 2 nR2 bins, B 2 (m 2 ), m 2 E [1 : 2 nR ' 2 }. 

• For each u n (l) and y n sequences, generate 2 nRl1 X" (l 7 mu) sequences according to Y[i=iP(xii\ui,yi). 
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B. Encoding at the encoder 

Given a (x n ,y n ) pair, the encoder first looks for an index I g [1 : 2 nR '] such that (u n (l), x n ,y n ) g Te , where 
7e stands for the set of jointly typical sequences. If there are more than one such I, it selects one uniformly at 
random from the set of admissible indices. If there is none, it sends an index uniformly at random from [1 : 2' li?, JH 
Next, it finds the index mn such that [pti (I, mn), u n (mio), x n , y n ) g 7? '. As before, if there is more than one, it 
selects one uniformly at random from the set of admissible indices. If there is none, it sends an index uniformly at 
random from [1 : 2 nRl1 ]. Finally, it sends out (mio,mn), where mio is the bin index such that u n (l) g Bi(mio). 
The total rate required is R. 

C. Decoding and reconstruction at Node 1 

Given (mm, mn), Node 1 looks for the unique 1 such that (u n (l),y n ) g % (n) and u n (l) g Bi(l). It reconstructs 
x n as x n (l, mu). If it failed to find a unique one, or if there is more than one, it outputs I = 1 and performs the 
reconstruction as before. 

D. Encoding at Node 1 

Node 1 sends an index rhi such that u n (l) g B^im-i)- This requires a rate of Ri- 

E. Decoding and reconstruction at Node 2 

Node 2 looks for the index I such that (u n (l), y n ) g Te"^ and I g ^2(7712). It then reconstructs x n according to 
x 2i = g2(u n (l)i, Zi) for i £ [1 : n]. If there is no such index, it reconstructs using 1 = 1. 

F. Analysis of expected distortion 

Using the typical average lemma in (6] Lecture 2] and following the analysis in J6| Lecture 3], it suffices to 
analyze the probability of "error"; i.e. the probability that the chosen sequences will not be jointly typical with the 
source sequences. Let L and Mn be the chosen indices at the encoder. Note that these define the bin indices Miq 
and Mi. Let M2 be the chosen index at Node 1. Define the following error events: 

1) £ := {{X n ,Y n ) i% [n) } 

2) £1 := {(U n (l),X n ,Y n ) £ T} n} } for all I £ [1 : 2 nRl ] 

3) £ 2 : = {(U n (l),X n ,Y n 1 Z n ) £ 7? n) } for all I g [1 : 2 nR >] 

4) £ 3 := {(U n (L),X n (L,m 1 i),X n ,Y n ) f Te {n} } for all mn g [1 : 2 nRl1 } 

5) £4 := {(U n (i),Y n ) g Te {n) } for some I ^ L and U n (l) g B 1 (M 10 ) 

6) £ 5 (M 2 ) := {(U n (l), Z n ) g T c {n) } for some l^L and U n (l) g ^2(^2) 
We can then bound the probability of error as 

5 i-l 

p e <p{U^}-E p ^ n (fl^)}- 

2 = j=0 

• P{^o} as n — > 00 by Law of Large Numbers (LLN). 

• By the covering lemma in |6, Lecture 3], P{£i PI £q} — > as n — > 00 if 

i?/ >/([/; X,F)+(e). 

• P{^2 H £f n £q} — > as n — >• 00 by the Markov relation [/ — (X, Y) — Z and the conditional joint typicality 
lemma Lecture 2]. 

• By the covering lemma in J6l Lecture 3], P{£^ n (Pl^=o ^j} — > as n — )■ 00 if 

R n >I(Xi;X\U,Y)+(e). 

1 For simplicity, we assume randomized encoding, but it is easy to see that the randomized encoding employed our proofs can be incorporated 
as part of the (random) codebook generation stage. 
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• From the analysis of the Wyner-Ziv Coding scheme (see Q or jfjj Lecture 12]), P{<?4 n (Hj—o ^j} ^ as 
n — > oo if 

R l -R w <I(U;Y)-(e). 

• For the last term, we have 

4 4 

P{£ 5 (M 2 ) n (f) £<)}= P{£ 5 (M 2 ) n (f) £<;) n {M 2 ^ M 2 }} 

j=o j=0 

4 

+ P{£ 5 (M 2 ) n ( f) £, c ) n {M 2 = M 2 }} 

4 

( = } P{£ 5 (M 2 ) n (fl 8f) n {M 2 = m 2 }} 

4 

= P{«? 5 (M 2 ) n (fl 8 C 3 ) n {M 2 = m 2 }} 
< P{f 5 (M 2 )n^ c }. 

Step (a) follows from the observation that (Hj=o ^j ) ^ {^2 7^ M2} = 0- The analysis of the probability of 
error therefore reduces to the analysis for the equivalent Wyner-Ziv setup with Z as the side information at 
Node 2. Hence, P{£ 5 (M 2 ) n (fl^o^ 9 )} as ?i ^ oo if 

Rl-R 2 <I(U; Z) -(e). 

Eliminating Ri in the aforementioned inequalities then gives us the required rate region. 
Achiev ability proof of Theorem \2\ 

As the achievability proof for the Triangular Source Coding Case follows that of the Cascade Source Coding 
Case closely, we will only include the additional steps required for generating R3 and analysis of probability of 
error at Node 2. The steps for generating R\ and i? 2 , and for reconstruction at Node 1 are the same as the Cascade 
setup. 

G. Codebook Generation 

• Fix p(x, y, z, u, v,xi)= p(x)p(y\x)p{z\y)p{u\x, y)p{x l \x, y, u)p(v\x, y, u). 

• For each u n (l), generate V n (ls), I3 G [1 : 2 nRs ], according to n"=iP( l; j|' lt i)- Partition the set of v n sequences 
into 2 nR3 bins, B 3 (m 3 ). 

H. Encoding 

• Given a sequence (x n ,y n ) and u n (l) found through the steps in the Cascade Source Coding setup, the encoder 
looks for an index £3 such that (it", v n (l, l 3 ),x n ,y n ) € 7e '■ If it finds more than one, it selects one uniformly 
at random from the set of admissible indices. If it finds none, it outputs an index uniformly at random from 
[1 : 2 nRs ]. The encoder then sends out 771,3 suc h that L3 £ ,63(7713). 

/. Decoding 

The additional decoding step is in decoding L3. Node 2 looks for the unique I3 such that (u n (l),v n (l, I3), z n ) <E 
Te and v n (l3) £ ,63 (M3). If there is none or more than one, it outputs 1113 = 1. 
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J. Analysis of Distortion 

Let L, Mn and M 3 be the indices chosen by the encoder. Note that these fix the indices Miq and M 2 . We 
follow similar analysis as in the Cascade case, with the same definitions for error events £q to £5. We also require 
the following additional error events: 

7) £ 6 := {(U n (L),V n (L,L 3 ),X n ,Y n ) £ % (n) }. 

8) £ 7 := {{U n (L),V n (L 7 L 3 ),X n ,Y n 7 Z n ) £ % (n) }. 

9) £ S (L) := {(U n (L),V n (L,l 3 ),Z n ) e % (n) ] for some l 3 + L 3 and l 3 e B 3 (M 3 ). 
To bound the probability of error, we have the following additional terms 

• By the covering lemma, P(£§ (~l £ 2 ) as n — > 00 if 

R 3 >I(V;X,Y\U) + (e). 



P(<?7 n £q) — > as noo from the Markov condition (V, [/) — (X, Y) — Z and the conditional joint typicality 
lemma. 

P{£ S (L) n £§(M 2 ) n £ 7 C n (0 J =0 5?))}. We have 

4 

P{f 8 (i)nf 5 c (Af 2 )nf 7 c n(f|^ c )} 

4 4 

= P{£ 8 (L) n £ 5 c (m 2 ) n £ c , n (f| £?) n {M 2 = M 2 }} + P{£ 8 (L) n £ 5 c (m 2 ) n £ 7 C n (f| £?) n {M 2 ^ M 2 }} 

i=o 3=0 
4 

= P{£ 8 (L) n ££(M 2 ) n £ 7 C n (f| £?) n {M 2 = M 2 }} 

4 

= P{£ 8 (L) n £ 5 c (a/ 2 ) n £ 7 C n (f| q) n {M 2 - m 2 }} 

3=0 

< P{£ 8 (L)n£l(M 2 )n£$} 

c = ] P{£s(i) n £ 5 c (A/ 2 ) n £ 7 C n {L = l}} 
= P{£ 8 (L) n £ C 5 (M 2 ) n £ 7 C n {L = l}} 

< P{£ 8 (L)n£ 7 c }. 

(a) follows from the observation that £g (M 2 ) n£ 7 C\{L ^ L} = ®. It remains to bound P{£ 8 (L) l~l£ 7 }. Note 
that the analysis of this term is equivalent to analyzing the setup where U n is the side information at Node 
and (U n ,Z n ) is the side information at Node 2. Hence, P{£ & (L) n £ 7 } ->• as n -> oo if 

Bs-JZa <7(V;Z|^)-(c). 

We then obtain the rate region by eliminating R 3 and Ri. 
Achievability proof of Theorem \3\ 

As with the case for the Triangular setting, the proof for this case follows the Cascade setting closely. We will 
therefore include only the additional steps. We have a change of notation from the Cascade setting. We will use 
Ui instead of U 



K. Codebook Generation 

• Fix p(x,y,z,ui,u 2 ,xi) = p(x,y, z)p(ui\x,y)p(xi\ui,x,y)p(u 2 \z,ui). 

• For each v%(l), generate 2 nRs U£(l 3 ) sequences, I 6 [1 : 2 nR3 \, each according to n™=i p( u 2i\uu)- Partition 
the set of U$ into 2 nIi3 bins, B 3 (m 3 ). 
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L. Encoding 



The additional encoding step is at Node 2. Node 2 looks for an index L 3 such that (u"(Z), uJf (£, £3), £ 7? . 
As before, if it finds more than one, it selects an index uniformly at random from the set of admissible indices. If 
it finds none, it outputs an index uniformly at random from [1 : 2 nRs }. It then outputs the bin index 7713 such that 
L 3 e B 3 {m 3 ). 



Additional decoding is required at Node 0. Node looks the index l 3 such that (u"(i)> ^(l, l 3 ), x n , y n ) € Te 
and t 3 g <B 3 (m 3 ). 

N. Analysis of distortion 

Let Scascade denote the event that an error occurs in the forward Cascade path. In addition, we define the 
following error events. 



. &rw-x{L) := {(U?(L),U2(L,h),Z n ) $ % {n) for all l 3 e [1 : 2 nR -]}. 

• £ TW -2(L) := {(U£(L), Ug(L, L 3 ), Z n , X n , Y n ) £ % {n) }. 

• Etw-z(L) := {(U?(L),U?(Lj 3 ),X n ,Y n ) e % {n) for some l 3 e B 3 (M 3 ),l 3 ? L 3 }. 
. P(£tw-i(L) n £ c Cascade ) = P(£tw-i(L) n £ c Cascade ) -> as n -> 00 if 

fl3>7(l72;^i) + (e). 

• P(f rW -2(7) n Scasea.de) = P (^TW-2(L) H £ Cascade ) -> as n ->■ oo by the strong Markov Lemma (8). 
. P(firHr_ 3 (L) n £ Cascade ) = P(£tw- 3 (L) n f& ascade ) -»- as n -> 00 if 

J R 3 -i?3</(C/ 2 ;X,y|C/i)-(e). 



Finally, eliminating R 3 and i?; gives us the required rate region. 
Achievability proof of Theorem 

The achievability proof for Two Way Triangular source coding combines the proofs of the Triangular source 
coding case and the Two-way cascade case. As it is largely similar to these proofs, we will not repeat it here. We 
will just mention that the codebook generation, encoding, decoding and analysis of distortion for the forward path 
from Node to Node 2 follows that of the Triangular source coding case, while codebook generation, encoding, 
decoding and analysis of distortion for the reverse path from Node 2 to Node follows that of the Two-way Cascade 
source coding case, with (U2, V) taking the role of Ui- 

Appendix B 
Cardinality Bounds 

We provide cardinality bounds for Theorems [T]|4] stated in the paper. The main tool we will use is the Fenchel- 
Eggleston-Caratheodory Theorem ifTOl . 

A. Proof of cardinality bound for Theorem\l\ 
For each x, y, we have 



We therefore have | 1 1 — 1 continuous functions of p(x,y\u). These set of equations preserves the distribution 
p(x,y) and hence, by Markovity, p(x,y,z). Next, observe that the following are similarly continuous functions of 



M. Decoding 




u 
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p(x,y\u) 

I(U; X, Y\Z) = H(X, Y\Z) - H(X, Y, Z\U) + H(Z\U), 
I{X;X X ,U\Y) = H(X\Y)-H{X\TJ) + H{X,%i,Y\U), 
Ed 1 (X,X 1 ) = y~]p(x,xi)d(x,xi), 

x,x 

Ed 2 (X,X 2 ) = ^2 p(x,y,u)d(x,g 2 (x,u)), 

x,y,u 

These equations give us 4 additional continuous functions and hence, by Fenchel-Eggleston-Caratheodory Theorem, 
there exists a U' with cardinality of |<-f||y| +3 such that all the constraints are satisfied. Note that this construction 
does not preserve p{x\), but this does not change the rate-distortion region since the associated rate and distortion 
are preserved. 

B. Proof of cardinality bound for Theorem^ 

We will first give a bound for the cardinality of U. We look at the following continuous functions of p(x,y\u). 

fj(px,Y\u(x,y\u)) =^2p(u)p(x,y\u) = p{x,y),Mx,y 

It 

I(U; X, Y\Z) = H(X, Y\Z) - H(X, Y, Z\U) + H(Z\U), 
I(X; X lt U\Y) = H(X\Y) - H{X\U) + H(X, X X ,Y\U), 
I{X, Y- V\U, Z) = H(X, Y, Z\U) - H(Z\U) - H(X, Y, V, Z\U) + H(V, Z\U), 
Edi(X,j£ x ) = y^p(x, x{)d{x, fi), 

X,X 

Ed 2 (X,X 2 )= ^2 p(x,y,u,v)d(x,g 2 (x,u)). 

x,y,u,v 

From these equations, there exists a U' with \U'\ < \X\\y \ + 4 such that the equations are satisfied. Note that the 
new U' induces a new V '. For each U' = u, consider the following continuous functions of p(x, y\u, v) 

p(x,y\u) = ^2p(v\u)p(x,y\v,u), 

V 

I(X,Y;V\U = u,Z) = H(X,Y\U = u,Z) - H(X 7 Y\V,U = u,Z), 
E{d 2 (X,X 2 )\U = u)=y^ j p(x,y,v\u)d(x,g 2 (x,u)). 

x,y,v 

From this set of equations, we see that for each U' = u, it suffices to consider V' such that |V'| < 1^113^1 + 1. 
Hence, the overall cardinality bound on V is |V| < (1^1 +4)(|A^||3 ; | + 1)- The joint p(x, y, z) is preserved due 
to the Markov Chain (V, U) - (X, Y) - Z. 

C. Proof of cardinality bound for Theorem\3\ 

The cardinality bounds on U\ follows similar analysis as in the Cascade source coding case. The proof is therefore 
omitted. For each U\ = u\, the following are continuous functions of p{z\u 2 ,u\), 

p{z\ui) = 'Y2p( u 2\ui)p{z\u 2 , m), 

I(U 2 ;Z\U 1 =u 1 ,X,Y)=H(Z\U 1 = u 1 ,X,Y)-H(Z\U 1 = u 1 ,U 2 ,X,Y), 
E(d s (Z,Z)\U 1 = mi) = 2J P(x,y,z,u 2 \u 1 )d(z,g3(x,y,ux,u 2 )). 

From this set of equations, we see that for each U\ = u\, it suffices to consider U 2 such that \U 2 \ < \Z\ + 1. Hence, 
the overall cardinality bound on U 2 is \U 2 \ < \U\\(\Z\ + 1). The joint p(x,y,z) is preserved due to the Markov 
Chains U x - (X, Y) - Z and U 2 - (Z, U x ) - (X, Y). 
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D. Proof of cardinality bound for Theorem^ 

The cardinality bounds follow similar steps to those for the first 3 theorems. For the cardinality bound for \U2\, 
we find a cardinality bound for each U\ = tii and V = v. Details of the proof are omitted. 

Appendix C 

Alternative characterizations of rate distortion regions in Corollaries [T]and[2] 

In this appendix, we show that the rate distortion regions in Corollaries Q] and [2] can alternatively be characterized 
by transforming them into equivalent problems found in J4j, where explicit characterizations were given. We focus 
on the Cascade case (Corollary [TJ, since the Triangular case follows by the same analysis. 

Figure [8] shows the Cascade source coding setting which the optimization problem in Corollary Q] solves. 



Node 




B B 



Fig. 8: Cascade source coding setting for the optimization problem in Corollary Q] Xi and X 2 are lossy 
reconstructions of A + B. 

In (4), explicit characterization of the Cascade source coding setting in Figure [9] was given. 



Node 




Y = X + Z Y — X + Z 



Fig. 9: Cascade source coding setting for the optimization problem in Corollary Q] X\ and X 2 are lossy 
reconstructions of X and Z is independent X. 

We now show that the setting in Figure [8] can be transformed into the setting in Figure [9] First, we note that 
for the setting in Figure [9] the rate distortion regions are the same regardless of whether the sources are (X, Y) or 
(X, aY) where a^O since the nodes can simply scale Y by an appropriate constant. 

Next, for Gaussian sources, the two settings are equivalent if we can show that the covariance matrix of (X, aY) 
can be made equal to the covariance matrix of (A + B,B). Equating coefficients in the covariance matrix, we 
require the following 

~2 2 , 2 
x = a a + B i 

2 2 
2/ 2 | 2 \ 2 

Solving these equations, we see that a = o%/{a\ + er^) and o\ = (a 2 B — ct 2 a\)/a 2 . Since (p\ — a 2 o~x) > 0, 
this choice of o\ is valid, which completes the proof. 
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